{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# EBM Feature/Term Importances"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook we show how to compute and interpret Overall Importances shown in InterpretML's Global Explanations for EBMs. We also show how to compute importances of a group of features or terms.\n",
    "\n",
    "Throughout the notebook we use _term_ to denote both single features and interactions (pairs)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train an Explainable Boosting Machine (EBM) for a regression task"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use the Boston dataset as a reference and train an EBM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.datasets import load_diabetes\n",
    "from interpret.glassbox import ExplainableBoostingRegressor\n",
    "\n",
    "dataset = load_diabetes()\n",
    "df = pd.DataFrame(dataset.data, columns=dataset.feature_names)\n",
    "df[\"target\"] = dataset.target\n",
    "\n",
    "train_cols = df.columns[0:-1]\n",
    "label = df.columns[-1]\n",
    "X = df[train_cols]\n",
    "y = df[label]\n",
    "\n",
    "seed = 42\n",
    "np.random.seed(seed)\n",
    "\n",
    "ebm = ExplainableBoostingRegressor(random_state=seed)\n",
    "ebm.fit(X, y) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explain the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "EBMs provide two different kinds of explanations: global explanations about the overall model behavior and local explanations about individual predictions from the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Global Explanation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Global Explanations are useful for understanding what a model finds important, as well as identifying potential flaws in its decision making or the training data. Let's start by computing and displaying a global explanation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from interpret import show\n",
    "\n",
    "ebm_global = ebm.explain_global(name='EBM')\n",
    "show(ebm_global)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The overall importance for each term is calculated as _the average absolute contribution (score) a term (feature or pair) makes when predicting across the training dataset._ This way of measuring term importance tends to favor terms which, on average, have large impact on predictions for many cases. The overall importance is not a measure of positive/negative -- it is a measure of how important each term is in the scores. For regression, these scores are represented in the same units as the y-axis of the feature graphs. For classification, the scores would be in logits."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Going beyond overall term importances, because EBMs are additive models we can measure exactly how each term contributes to a prediction. Let's take a look at the graph of the term, `bp`, by selecting it in the drop-down menu.\n",
    "\n",
    "![Global Explanation - LSTAT](../assets/importance_notebook_global_lstat.png)\n",
    "\n",
    "The way to interpret this is that if a new datapoint came in with `bp` = 0.1, the model adds about +33.1 to the final prediction. However, for a different datapoint with `bp` = 0.13, the model would now add approx. +36.7 to the prediction.\n",
    "\n",
    "To make individual predictions, the model uses each term graph as a look up table, notes the contribution per term, and sums them together with the learned intercept to make a prediction. In regression, the intercept is the mean target (label) of the training set, and each term adds or subtracts to this mean. In classification, the intercept reflects the base rate of the positive class on a log scale. The gray above and below the graph shows the confidence of the model in that region of the graph."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Local Explanations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the full breakdown of a prediction on a single sample with Local Explanations. Here's how to compute the prediction breakdown for the first sample in our dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from interpret import show\n",
    "index = 0\n",
    "show(ebm.explain_local(X[index:index+1], y[index:index+1]), 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a look at the prediction by selecting it in the drop-down menu.\n",
    "\n",
    "![Local Explanation](../assets/importance_notebook_local_exp.png)\n",
    "\n",
    "The model prediction is 188.50. We can see that the intercept adds about +151.9, `bp` subtracts about 0.02, and `age` adds about 0.04. If we repeat this process for all the terms, we'll arrive exactly at the model prediction of 188.50."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Viewing _all_ term importances"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Due to space limitations in our graphs, the term importance summary only shows the top 15 terms. To view the overall importances of all terms of a trained EBM - the scores shown in the global explanation summary - we use `term_importances()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "importances = ebm.term_importances()\n",
    "names = ebm.term_names_\n",
    "\n",
    "for (term_name, importance) in zip(names, importances):\n",
    "    print(f\"Term {term_name} importance: {importance}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that mean absolute contribution isn't the only way of calculating term importances. Another metric our package provides is the `min_max` option, which computes the difference between the `max` (the highest score on the graph) and `min` (the lowest score on the graph) values for each term. Term importance measured with `min_max` is a measure of the maximum impact a term can have, even though it might have this amount of impact on very few cases, whereas `avg_weight`(the default parameter) is a measure of typical (average) contribution of a term across all cases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "importances = ebm.term_importances(\"min_max\")\n",
    "names = ebm.term_names_\n",
    "\n",
    "for (term, importance) in zip(names, importances):\n",
    "    print(f\"Term {term} importance: {importance}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature/Term Group Importances"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We provide utility functions to compute the importances of groups of features or terms and, optionally, append these importances to the global feature attribution bar graph. Note that shape function graphs are not generated for groups of features/terms, just their overall importance is shown on the Summary.\n",
    "\n",
    "Grouping terms and then calculating and displaying their importance does not change the model and the predictions it makes in any way -- group importances are just a method for computing the importance of groups of terms in addition to the importances of individual terms that are already calculated. As you'll see in the examples below, it's OK for features/terms to overlap in different groups."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Computing group importances"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use the Adult dataset and train an EBM for a classification task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from interpret.glassbox import ExplainableBoostingClassifier\n",
    "\n",
    "df = pd.read_csv(\n",
    "    \"https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data\",\n",
    "    header=None)\n",
    "df.columns = [\n",
    "    \"Age\", \"WorkClass\", \"fnlwgt\", \"Education\", \"EducationNum\",\n",
    "    \"MaritalStatus\", \"Occupation\", \"Relationship\", \"Race\", \"Gender\",\n",
    "    \"CapitalGain\", \"CapitalLoss\", \"HoursPerWeek\", \"NativeCountry\", \"Income\"\n",
    "]\n",
    "\n",
    "train_cols = df.columns[0:-1]\n",
    "label = df.columns[-1]\n",
    "\n",
    "X = df[train_cols]\n",
    "y = df[label].apply(lambda x: 0 if x == \" <=50K\" else 1)\n",
    "\n",
    "seed = 42\n",
    "np.random.seed(seed)\n",
    "\n",
    "adult_ebm = ExplainableBoostingClassifier(random_state=seed)\n",
    "adult_ebm.fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We then create a list of terms -- single features or interactions -- as our group and compute its importance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from interpret.glassbox.ebm.research.group_importance import *\n",
    "\n",
    "social_feature_group = [\"MaritalStatus\", \"Relationship\", \"Race\", \"Gender\", \"NativeCountry\"]\n",
    "importance = compute_group_importance(social_feature_group, adult_ebm, X)\n",
    "print(f\"Group: {social_feature_group} - Importance: {importance}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example we create a group with five terms and compute its importance. Similar to single feature importances, we interpret this score as _the average absolute contribution this group of terms makes when predicting across the training dataset._ Note that for each prediction, the contribution of each term in the group will be added before taking the absolute value. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also have the option to create a global explanation containing the group importance or append it to an existing explanation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_global_exp = append_group_importance(social_feature_group, adult_ebm, X)\n",
    "show(my_global_exp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The importance of `social_feature_group` is about 1.30, which is higher than the importance of any individual feature/term:\n",
    "\n",
    "![Global Explanation - Social Feature Group](../assets/importance_notebook_social_group.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could make this type of comparison between different groups too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "education_feature_group = [\"Education\", \"EducationNum\"]\n",
    "relationship_feature_group = [\"MaritalStatus\", \"Relationship\"]\n",
    "social_feature_group = [\"MaritalStatus\", \"Relationship\", \"Race\", \"Gender\", \"NativeCountry\"]\n",
    "my_global_exp = append_group_importance(social_feature_group, adult_ebm, X)\n",
    "my_global_exp = append_group_importance(education_feature_group, adult_ebm, X, global_exp=my_global_exp)\n",
    "my_global_exp = append_group_importance(relationship_feature_group, adult_ebm, X, global_exp=my_global_exp)\n",
    "show(my_global_exp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The importance of `education_feature_group` is about 0.52, higher than each of its individual terms but smaller than some individual terms such as `Age`. Remember, creating groups of features/terms does not, in any way, change the model and its predictions, it only allows you to estimate the importance of these groups.\n",
    "\n",
    "This graph, for example, suggests that features related to relationships are more important than features reated to education.\n",
    "\n",
    "![Global Explanation - Education Group](../assets/importance_notebook_education_group.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also compare a group we are interested in (e.g. `social_feature_group`) with a group of all other reamining terms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "social_feature_group = [\"MaritalStatus\", \"Relationship\", \"Race\", \"Gender\", \"NativeCountry\"]\n",
    "all_other_terms = [term for term in adult_ebm.term_names_ if term not in social_feature_group]\n",
    "\n",
    "my_global_exp = append_group_importance(social_feature_group, adult_ebm, X)\n",
    "my_global_exp = append_group_importance(all_other_terms, adult_ebm, X, group_name=\"all_other_terms\", global_exp=my_global_exp)\n",
    "show(my_global_exp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `all_other_terms` has the highest importance score, followed by  `social_feature_group`. \n",
    "\n",
    "![Global Explanation - All Other Group](../assets/importance_notebook_all_other_group.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's even possible to create a group with all terms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_terms_group = [term for term in adult_ebm.term_names_]\n",
    "mew_global_exp = append_group_importance(all_terms_group, adult_ebm, X, group_name=\"all_terms\")\n",
    "show(mew_global_exp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we also expose a function to compute the importances of a group of terms as well as all the model's original terms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_dict = get_group_and_individual_importances([social_feature_group, education_feature_group], adult_ebm, X)\n",
    "for key in my_dict:\n",
    "    print(f\"Term: {key} - Importance: {my_dict[key]}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
